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Abstract —We consider the discrete memoryless degraded 
broadcast channels. We prove that the error probability of 
decoding tends to one exponentially for rates outside the capacity 
region and derive an explicit lower bound of this exponent 
function. We shall demonstrate that the information spectrum 
approach is quite useful for investigating this problem. 



I. The Capacity Region oe the Degraded 
Broadcast Channels 

Let X, y, Z be finite sets. The broadcast channel we study 
in this paper is defined by a discrete memoryless channel 
specified with the following stochastic matrix: 

W = {W{y, z\x)}i^^^y^^)^xxyxz- ( 1 ) 

Here the set X stands for a set of channel input. The sets y 
and Z stand for sets of two channel outputs. Let X" be a 
random variable taking values in Y". We write an element of 
T’" as x" = xiX 2 - ■ ■ Xn- Suppose that X" has a probability 
distribution on Y” denoted by px^ = {px^{x‘^)}xr<.^x^- 
Similar notations are adopted for other random variables. Let 
Y" € Y" and Y" G Y" be random variables obtained as the 
channel output by connecting X" to the input of channel. We 
write a conditional distribution of on given X" as 

In this paper we deal with the case where the components 
W{z,y\x) of W satisfy the following conditions: 

W{y,z\x) = Wi{y\x)W2iz\y). (2) 

In this case we say that the broadcast channel W is de¬ 
graded. The degraded broadcast channel (DBC) is specified 
by (Wi, W 2 ). Transmission of messages via the degraded BC 
is shown in Lig. [T] Let and be uniformly distributed 
random variables taking values in message sets and C„, 
respectively. The random variable is a message sent to the 
receiver 1. The random variable is a message sent to the 
receiver 2. A sender transforms X„ and into a transmitted 
sequence X" using an encoder function and sends it to 
the receivers 1 and 2. In this paper we assume that the encoder 
function is a stochastic encoder. In this case, is a 
stochastic matrix given by 

where (x"|fc, Z) is a conditional probability of x" € X" 
given message pair (fc, 1) G x £„. The joint probability 


Fig. 1. Transmission of messages via the degraded BC. 


mass function on /C„ x £„ xX" xV" X-Z" is given by 
Pr{(X„, X", Y”, Y”) = (fc, I, x", y”, z^)} 

where |/C„| is a cardinality of the set /C„. The decoding 
functions at the receiver 1 and the receiver 2, respectively, 
are denoted by -0^"^ and Those functions are formally 

defined by : Y" -A ■ -2^" ^n- The average 

error probabilities of decoding at the receivers 1 and 2 are 
defined by 

= Pr{r/>(")(Y") ^ X„}, 

= Pr{4”^(Y") ^ LJ. 

Purthermore, we set 

= Pt{4^\y^) ^ or ^ T„}. 


It is obvious that we have the following relation. 

p(n) <pW+p(-), 


( 3 ) 


Lor fc G X„ and I G set Vi{k) = {y” : = 

fc}, 222(0 = {z” : 44 = 1}- The families of sets 

{I2i(fe)}fegx;„ and {'D2{l)}i^Cn are called the decoding re¬ 
gions. Using the decoding region, Pi"^ can be written as 
1 


p(") _ 


\)C„\\Cr, 


E 


E 


1 


{k,l)GK.nXCn (a:",y”•.^")gA"xy'"xZ" 
y^^-Dlik) or 


X (x" |fc, ObPf (2/” k”)TY 2 " (z" |y"). 


Set 


p(™) = p(n) (^(n) ^ ^(n) ^ ^(n) ^ A ^ _ p(„) ^ ^(„) ^ ^(„) ^ ^ 















(n) 

The quantity Pc is called the average correct probability 
of decoding. For given (ei,£ 2 ) G (0,1)^, a pair is 

{ei,£ 2 )-achievable if there exists a sequence of triples 
}“=i such that 




liminf — log |/C„| > i?i, liminf — log |£„| > i? 2 - 

n—)-co 77, n^oo 77, 


The set that consists of all (ei,e 2 )-achievable rate pair is 
denoted by Cdbc{£i,£ 2 \Wi, W 2 ), which is called the capacity 
region of the DEC. We can define another capacity region 
based on the error probability i’l ^ '^ 2 )- 

given e G (0,1), a pair (i?i, i? 2 ) is e-achievable if there exists 
a sequence of triples such that 

lim inf — log |/C„| > Ri, lim inf — log |£„ | > i? 2 ■ 

n—^oo 77, n—^oc 77, 


The set that consists of all e-achievable rate pair is denoted 
by CDBc(£|bFi, W 2 ). It is obvious that for 0 < ei + £2 < 1, 
we have 


Set 

C(p) = {(i?i,i?2) >0, 

Ri<Ip{X;Y\U),R2<Ip{U;Z)}. 

C{Wi,W2)= U C{p). 

pev{Wi,W2) 

We can show that the above functions and sets satisfy the 
following property. 

Property 1: 

a) The region C(Wi,W 2 ) is a closed convex set of The 
region C(Wi, W 2 ) is a closed convex subset of where 

= {(i?i,i?2) : i?i > 0,i?2 > 0}. 

b) The region C(IFi, W 2 ) can be expressed with a family of 
supporting hyperplanes. To describe this result we define 
the set of probability distribution p = puxYZ of {U, X, 
Y, Z) gU xX xy xZ by 

V,^{Wi,W2) = {p:\U\ < IT”!, 

PY\X = Wi,pz\Y = W2,U -H- X -H- y -H- Z}. 


CdBc(£ 1 j £2111^1, 1^2) ^ CDBc(ei + £ 2 \Wi,W 2 )- 

We set 

Cdbc(IPi) IP 2 ) = Pi Cdbc(£|IP"ij IP 2 ), 

Ee(o,i) 

which is called the capacity region of the DEC. The two 
maximum error probabilities of decoding are defined by as 
follows: 

= , PrW!”>(P") ^ t|i<„ = I;}, 

(fc, t) ^/Cn X £77, 

p(n) _ p(n) , („) An)-. 

^e,m,2 ~ -^e,m,2VV^ ’ r2 ) 

= maxPrl^^^C^"”) ^ /|L„ = 1}. 

Eased on those quantities, we define the maximum capacity 
region Cm,DBc(ei,£ 2 |bFi, 1 ^ 2 ) in a manner quite similar to 
the definition of Cdbc(£ii £^ 2 |I^i, ^ 2 )- To describe previous 
works on Cbbc{Wi, W2) and Cm, dbg( ei, £^211^1, IT 2 ), we in¬ 
troduce an auxiliary random variable U taking values in a finite 
set U. We assume that the joint distribution of {U, X, Y, Z) is 

PuxYz{u, X, y, z) = Pu{u)px\u{x\u)Wi {y\x)W2{z\y). 

The above condition is equivalent to U GG X gg Y Z. 
Define the set of probability distribution p = puxYZ of (C/, 
X, Y, Z) gU xX xy xZ by 

r(Wi,W2) = {p: 1^1 <1X1 + 1, 

PyIX = ITli PzjY = IT 2 , R GG X G->- Y <->■ Z}. 


We set 

C^f^)(Wi,W2) 

= max {pIp(X; y|17) + Ip(U; Z)} , 

Csh(Wi,W2) 

= f^{(Ri,R2) ■■ pRi + R 2 < C(^HWi,W2)}. 

/i>0 

Then we have the following 

C(fPi,W'2) = Csh(FFi,W'2). 

Property [T] is a well known result. We omit the proof of this 
property. The broadcast channel was posed and investigated 
by Cover [1]. Eergmans ^ proved that CiVVi, W 2 ) serves as 
an inner bound of CDBc(bFi, IT 2 )- Gallager 13, Ahlswede and 
Korner a, proved that the inner bound C(IFi,lT 2 ) is tight, 
thereby establishing the following theorem. 

Theorem 1 (Gallager Ahlswede and Korner ^): 

For any DEC {Wi, W 2 ), we have 

CDBciWi,W2)=CiWi,W2). 

The strong converse theorem was proved by Ahlswede et 
al. 0. Their result is the following: 

Theorem 2 (Ahlswede et al. m): For each fixed 
(£i,£ 2 ) G (0,1)^ and any DEC (Wi, W 2 ), we have 

Cm,DBc(ei: £211^1, IT 2 ) = Cdbc(ITi, IT 2 ). 

Their method used to prove the strong converse theorem was 
extended to the method called the image size characterization 
by Csiszar and Korner 0. 


(n) 

To examine an asymptotic behavior of Pc for rates out¬ 
side the capacity region C(Wi, W 2 ), we define the following 
quantity. 




A 


min logP(”)(^(”\4"\V’2”^)) 


(1/n) log |£„|>it2 


G(i?i,i?2|fPi,VP2) = lim G(”)(i?i,i?2|fPi,M^2). 


Our main aim is to find an explicit In this paper we derive an 
explicit lower bound of G(i?i, i? 2 |(fPi, W 2 ) that is positive if 
and only if (i?i, i? 2 ) ^ C(Wi, W 2 ). 


II. Main Result 

In this section we state our main result. Define 

uj^gf^\x,y,z\u) 

A , qY\x{y\x) , , qz\u{z\u) 

= ^log- . I . + log . X , 

qY\u[y\uj qz{z) 

A(M.^)(xrz|G) 

= X] qux{u,x)qY\x{y\x)qz\Y{z\y) 

{u,x,y,z)^UxX xyx2 

X exp|Aa;l'"l(a:,?/,z|u)|, 

nli^’^\xYZ\u) = iogAi^’^i(xrz|G), 

n^i^’^)(Wi,W 2 )= max ni>^’^^XYZ\U), 

qeVsh{Wi,W2) ^ 

+i?2|VPl,VP2) 

A X{yRi + R 2 )-n^>^'^\Wi,W 2 ) 

l + 2A + A/i 
F(i?i,i?2|VPl,VP2) 

= sup +i? 2 |VPl,IP 2 ). 

fi,X>Q 

We can show that the above functions and sets satisfy the 
following property. 

Property 2: 

a) For each q S V{Wi,W 2 ), n^q^’^\XYZ\U) is a mono¬ 
tone increasing and convex function of A > 0. 

b) For every q € 7^sh(lPi, W 2 ), we have 

lim — - - 

A^+o A 

= pIq{X;Y\U)+Ig{U;Z). 

c) If (i?i,i? 2 ) ^ C(IPi,VF 2 ), then we have F(i?i, i? 2 | 
Wi,W2) > 0 . 

Proof of Property |2] is given in Appendix |B] Our main 
result is the following. 

Theorem 3: For any degraded BC (PFi, IF 2 ), we have 

G{RyR2\WyW2) > F{RyR2\WyW2). (4) 

Proof of this theorem will be given in Section |III] It follows 
from Theorem |3] and Property |2] part c) that if (i?i,i? 2 ) 


is outside the capacity region, then the error probability of 
decoding goes to one exponentially and its exponent is not 
below F{Ri, i? 2 |IFi, W 2 )- From this theorem we immediately 
obtain the following corollary, which partially recovers the 
strong converse theorem by Ahlswede et al. 0. 

Corollary 1: For each pair {£i,£ 2 ) € (0,1)^ satisfying £1 + 
£2 < 1, we have 

Cm,DBc(£l: e2|fPl, fP2) 

= CDBc(ei: e2|fPi, W 2 ) = CDBc(ei + e2|fPi, fP2) 

= Cdbc(M"i,VF2) =C(IPi,VF2). 

In particular, for each e: G (0,1/2), we have 

CDBc(e,e|VF) = CDBc( 2 e|VFi, VF2) = C(VFi, VF2). 

The exponent function at rates outside the channel capacity 
was derived by Arimoto m and Dueck and Korner 0 . The 
techniques used by them are not useful to prove Theorem |3] 
Some novel techniques based on the information spectrum 
method introduced by Han 0 are necessary to prove this 
theorem. 


III. Proof of the Results 


We first prove the following lemma. 

Lemma 1: For any p > 0 and for any ((/?("), ^ 2 "^) 

satisfying (1/n) log |/C„| > i?i, (1/n) log |£„| > i? 2 - we have 




1 W^{Y'^\X'^)W^{Z'^\Y'^) 

~ qY^z^\LA^^,Z^\Ln) 

„/l, Pzr^\LSZ'^\Ln) 

R2< - log - i - 


-f 77 


+ q\+2e-^\ 


( 5 ) 

(6) 


In (|5]), we can choose any conditional distribution gyn z''\l„ on 
yn ^ 2 ,"^ given G £„. In ® we can choose any probability 
distribution gyn on Z”. 

Proof of this lemma is given in Appendix |C] 

For f = 1, 2, • • •, n, set 


Ut=CnX X Z*-\Vt = FnX Z*-\ 

Ut = (L„,F‘-\Z‘-i) G = (L„,Z‘-i) G Vt, 

ut = {l,y*~^,z*~^) G Ut,vt = G Vt- 


For each f = 1, 2 • • •, Z, let kj be a natural projection from Ut 
onto Vt- Using Kt, we have Vt = Kt{Ut), t = 1, 2, • • •, n. For 
each f = 1, 2, • • •, n, let Q{Ut x A’x 3^ x Z) be a set of all 
probability distributions on 

Utx X xy X z = CnX X xy* X zK 


For t = 1, 2, ■ • •, n, we simply write Qt=Q{Ut xX x 3^ x Z). 
Similarly, for f = 1, 2, • • •, n, we simply write qt = qutXtYtZt 
G Qt- Set 

n n 

= Y[Qt = Y[Q{Kt X X X y X z), 

9 ” = {9t}r=i G 2”. 








By Lemma [T] and some computations we have the following 
lemma. 

Lemma 2: For any rj > 0, for any ) satis¬ 

fying (1/n) log |/C„ | > Ri, (1/n) log |£„| > R 2 , and for any 

qU g Qn^ jj^yg 


O / 1 Wl{Yt\Xt) , 

r? 




'7Zt(^t) 


2 e 


— 717] 


(7) 


where for each f = 1 , 2 , • • •, n, the conditional probability dis¬ 
tribution qYt\Ut the probability distribution qz^ appearing 
in the first term in the right members of (I7]i are chosen so 
that they are induced by the joint distribution qt = qutXtYtZt 

s Qt- 

Proof of this lemma is given in Appendix |D] 

To evaluate an upper bound of (|7]i in Lemma |2] We use the 
following lemma, which is well known as the Cramer’s bound 
in the large deviation principle. 

Lemma 3: For any real valued random variable Z and any 
0 > 0 , we have 


Pt{Z > a} < exp [— (Ao — logE[exp(0Z)])]. 

Here we define a quantity which serves as an exponential 
upper bound of LetkF 2 ) be 

a set of all probability distributions pl^X’^y^Z’' on £„ XT’” 
x>’” xZ” having the form: 


Pl^x^Y’^z^ (/, x”, y”, z") 

n 

= PL^{l)W_PxpL^xt-^[xt\l,x*~^)Wi{yt\xt)W2{zt\yt). 


For simplicity of notation we use the notation p^'^'> for 
Pl„x-‘Y^z-‘ e {Wi,W 2 ). We assume thzi pu^XtYtZt = 
PL„XtY*Zt is a marginal distribution of For t = 

l, 2 ,---,n, we simply write pt = PUtXtYtZf For G 
T’("i(kFi,H/ 2 ) and g" G Q”, we define 




= logE („) 


n 


qt\uSYt\Ut)q%SZt) 


where for each f = 1 , 2 , • • •, n, the conditional probability dis¬ 
tribution qYt\Ut probability distribution qzt appearing 

in the definition of are chosen so that 

they are induced by the joint distribution gt = qutXtYtZt C Qt- 
Here we give a remark on an essential difference between 
p(.n) g (ILi, kF 2 ) and g” G Q". For the former the n 
probability distributions pt, t = l, 2 ,---,n, are consistent 
with since all of them are marginal distributions of 

On the other hand, for the latter, g" is just a sequence 
of n probability distributions. Hence, we may not have the 
consistency between the n elements gt, f = 1 , 2 , • • •, n, of g". 


By Lemmas |2] and 12 we have the following proposition. 
Proposition 1: For any p, 9 > 0, any g” G Q", and any 
satisfying 


- log \JCn\ > Rl, - log \Cn\ > i?2, 
n n 


( 8 ) 


we have 

p(n)(^(n)^^ 0 )^^(n)) 


+ R 2 ) - 

1 + 9 +Op 


< 3 exp < —n 


Proof: Under the condition ([2), we have the following chain 
of inequalities: 


1 

Rl < -Y^^og 


Wi{Yt\Xt) 

tr"’^Ypumut) 


V, 


i ?2 < - E log ^ ^ ^ ^ 




(IZtiZt) 


< PL,^X^Y''Z^\ pRl + R 2 — [p + 1)?7 


< - ^ log 

rj 




(b) 

< 


Wr{Yt\Xt)pzpvXZt\yt) 

<l^puYYt)q%,{Zt) 
expj^n|—0(yi?i -I- R 2 ) + 9{p + l)p 
„(x"y"Z"|L„) 


-f 3e-"’' 


3e- 


(9) 


Step (a) follows from Lemma|2 Step (b) follows from Lemma 
[2 We choose rj so that 

— p = -9{pRi + R 2 ) + 9{p + 1)77 

+ ( 10 ) 

Solving (Unii with respect to 77 , we have 


77 = 


9{pR, + R 2 ) - 


1 + 9+ 9p 
For this choice of 77 and we have 
pl”) < 3e-”’' 


= 3 exp < —77 


9{pR^+R2) - 

1 + 9+ 9p 


completing the proof. 
Set 


d-'"'^\Wx,W2) 


A 


1 


= sup max min n(^^Y"Z^lLn)- 

/ipMep(-HWi.W2)YeQ- n 
















By the above definition of R 2 \Wi, W 2 ) and Propo¬ 

sition [1] we have 


n ■ 

Then from (fTTT i. we obtain the following corollary. 
Corollary 2: For any 0 > 0, /i > 0, we have 


( 11 ) 


G(i?i,i?2|H^i,VF2) > 


e{^lRl + R2)-n''^'^\wl,W2) 

1 + 0+ en 


We shall call W 2 ) the communication potential. 

The above corollary implies that the analysis of 
1 ^ 1 , 11 ^ 2 ) leads to an establishment of a strong converse 
theorem for the degraded BC. 

The following proposition is a mathematical core to prove 
our main result. 

Proposition 2: For 9 S (0,1), set 


A = 


9 

1-9 


^0 


A 

1 + A' 


( 12 ) 


Then, for any 9 G (0,1), we have 

1 -f- A 

Proof of this proposition is in Appendix |E] The proof is not 
so simple. We must introduce a new method for the proof. 
Proof of Theorem]^ For 0 € (0,1), set 


A = 


9 

1-9 


^0 


A 

1-f A' 


( 13 ) 


Then we have the following: 


G(i?i,i?2|M"i,W'2) 

W 9{nRi + R2) -n‘''"’^\Wi,W2) 

1 + ^(1 + fi) 

(b) 

i + i^(i + m) 

_ x{fj.Ri + R 2 ) - n^>^’^\WuW2) 
1 + A + A(1 + fi) 

= F^>^'^\fiRi+ R2 \Wi,W2). 


Step (a) follows from Corollary |2l Step (b) follows from 
Proposition |2] and (fOl l. Since (fT4li holds for any A, /i > 0, 
we have (|4]i in Theorem [3] ■ 


IV. Concluding Remarks 

For the DBC, we have derived an explicit lower bound of 
the optimal exponent function on the correct probability of 
decoding for rates outside the capacity region. Our method for 
the DBC can also be applied to the derivation of an explicit 
lower bound of the optimal exponent function outside the 
capacity region for the asymmetric broadcast channels(ABCs)( 
or said the broadcast channels with degraded message sets) 
investigated by 0, Qoi-iia. In fact the author El succeeded 


deriving an explicit lower bound of the exponent function that 
is positive for rates outside the capacity region of the ABC. In 
the case of ABC, some additional techniques are also needed. 
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Appendix 

A. Cardinality Bound on Auxiliary Random Variables 

We have the following lemma. 

Lemma 4: For each integer n > 2, we define 

A 

= max 

Q—QUXY z'U XZ, 

‘Jy |x—fUi ,qz\Y—^ 2 -, 

\u\<\c+\yr-^ 

= max nf^\XYZ\U). 

q—quxYZ'U-^Xi^Y oZ, 
qY\x=Wi,qz\Y=W2, 

\U\<\X\ 

Then we have 

Proof: We bound the cardinality \U\ of U to show that the 
bound \IA\ < \X\ is sufficient to describe (Wi, W 2 ) and 












(14) 


{Wi,W 2 )- Observe that 

^x{x) = ^ qu{u)qx\u[x\u), 
uGU 

uGU 

where 


(15) 


= X! qx\u{x\u)Wi{y\x)W 2 iz\y) 
{x,y,z)G^xyxZ 

X exp|Aa;('")(a;,y,z|u)| 

For the quantities qz{') contained in the forms of 
{qx\u{'\'>^))! u GU, we regard them as constants under (fT4l) . 
For each u G U, C^^’^H9x|c/('l^)) continuous functions 
of qx\u{'W)- Then by the support lemma, 


\U\<\X\-1 + 1 = \X\ 


is sufficient to express lAj — 1 values of (O and one value 
of O. ■ 


B. Proof of Property 

In this appendix we prove Property |2 
Proof of Property \2} We first prove part a) and b). For 
simplicity of notations, set 

a = {u,x,y,z),A = {U,X,Y,Z),A = U X X x y x Z, 

= p(a),0('^-^)(XFZ|C/) =^(A). 

Then we have 


fi(f^A)(^XYZlU) = ^(A) 


log 


aeA 


By simple computations we have 


C'(A) 

r(A) 






= 


(16) 


X 


E gA(a)gA(&) ~ 

a,bGA 


■ (17) 


From dull, it is obvious that ^"(A) is nonnegative. Hence 
flq^’'^\xYZlU) is a convex function of A. It follows from 
(fT^ that for each q G V{Wi, W 2 ), we have 


^'(0) = ^qA{a)p{a) 

a 

= yilAX-,Y\U)+Iq(U\Z)>Q. (18) 


Hence we have the part b). Since ^'(0) > 0 and ^"(A) > 0, 
we have $'(A) > 0 for A > 0. Hence for each q G V[Wi^ W 2 ), 
^\l^'^\XYZ\U) is monotone increasing for A > 0. Next we 
prove the part c). We assume that ^ C(lFi,lT 2 ), 


then by Property [T] part b), there exist p* > 0 and e > 0, such 


that 

+i?2 > +e. (19) 


Set 


C(A) = ^(A) - A [4(X; Y\U) + /,(C/; Z) + 


Then we have the following: 


C( 0 ) = 0 , C'(0) = C"(A) = r(A) > 0 . ( 20 ) 

It follows from (EOll that there exists iy(e) > 0 such that we 
have C(A) < 0 for A S (0, v{e)]. Hence for any A G (0, v{e)\, 
for any /r > 0, and for every q G Psh(lT"i, VF 2 ), we have 


nM([/xYZ) < A (ylAX;Yll7) + Ig([/; z) + 0 .(21) 
From (|2TI) . we have that for any A G (0,z^(e)] and for any 

p > 0, 


n(^’^^(Wi,W2) 

= max ni'^’^hUXYZ) 

gePsh(Wi,W2) ^ 


< A 

= A 
= A 


max {yIg(X;Yll7)+Ig(U;Z)} + ^ 

_^€’Psh(VV'i ,W2) Z 

max {ylAX; y|17) + /,([/; Z)} + ^ 

_<?€'Psh(VFi,VK2) 2 

C^^\Wi,W2)X^ . 


( 22 ) 


Under (fT^ and (l22li . we have the following chain of inequal¬ 
ities: 

F{R^,R2\Wi,W2) 

= supsupFl^’'^l(pi?i -I-i?2|PFi, IU 2 ) 

A>0 /i>0 

> sup -f i? 2 |lUi,W 2 ) 

Ae(0,!/(e)] 

A(y*Ri+R2)-Al^^’'^\Wi,W2) 
Ae(o,y(e)] 1 -f 2A -f Ap* 

(a) M* 7 ?i + i? 2 -C('^*^(l^i,VF 2 )-^ 

> sup A-—-;- — 

Ae(o.!/(e)] 1 -I- 2A -I- Ap,* 

(b) 1 Ae 

> sup - • -—-;- 

Ae(o.!^(e)] 2 1 -I- 2A -I- Ap* 

= i __> 0 . 

2 1 -f 2i/(e) -f v(e)p* 

Step (a) follows from (l22li . Step (b) follows from (fTOl l. ■ 


C. Proof of Lemma [7] 

In this appendix we prove Lemma [1] 

Proof of Lemma \J} For I G Cn, set 

Ai{l) = {(a:",y",z") : W^(z^\y^)Wf{y^\x^) 

Al2(/) = : Pz"|L„(^”IO > ICAe-^^qzAz^}, 

Ail) = Ai(l)r\A2(l). 


















Then we have the following: 


a-nri 


< 


\Cr. 




p(n) _ 


\ICn\\Cr, 


E 


E 1 


i6£„ ksKr. 


(fc,0eK:„X-C„ (a:",y",z")e^( 0 , 

y"Gr>i(fc),z"Gr> 2(0 


I/:. 


E ( U ^l(^) 


X (x" I fc,/) irr (y” la:”) 1 ^ 2 ” (z” I y”) 


iG£„ 


vfcGKn 


< 


I/:. 


Ei = 


iG£„ 


(fc.0e/C„x£„ (x'‘,y",z")G^‘=(0: 

!/”Gl>i(fc),z”'GX>2(0 


X (x" Ifc,/) VF”(y" |x”) W 2 ”(-2” Iy") 

< E 


Next we prove A 2 < e We have the following chain of 
inequalities: 

1 


Aq = 


i=0,1.2 




where 


An = 


A 1 


■ E E 

(fc.0e/C„x£„ 

y"GX>i(fc).z'*GX> 2(0 
X |£n|9zn ( 2 ") 


1 


E E 


|A^n||A„l (a;".y",z")G^(/) 

xpx"Y"Z"|jf„,L„(a;”, y”, z”|A:, /), 


xPiv„A"y"Z"|L„(^, a:”, y”, 2:”|0 


< 


1^ 


^E E E E 


A A 1 

A,; = 


\ICn\\C„ 


E 


E 1 


(k,l)GfCnX-Cn {x'^,y'^,z'^)GA^(l), 
y^GVi{k),z^€V2{l) 


xpx"y'*Z'‘|iv„,L„(a:”, y”, z”|fc, 0 
for i = 1, 2. 


By definition we have 

Aq 


1, , 1, lFT(y"|A”) 

- log \ICn\ < - log- + y, 

n n qYn\L^{Y^\L„) 

1, I/- 1/ li Pz^\L,^iZ^\Ln) \ 

-log|An| < -log-^—— - +p'>. (23) 

n n y^n(Z”) J 

From (|2^ . it follows that if ) satisfies (1/n) 

log|/C„| > i?i, (l/n)log|A„| > R 2 , then the quantity Aq is 
upper bounded by the first term in the right members of (l6]l 
in Lemma [T] Hence it suffices to show A^ < = 1,2 

to prove Lemma [1] We first prove Ai < We have the 

following chain of inequalities: 


iGCn z"-eT> 2 il), keK„ (x”',y”')GX"xy" 

x|£„|ljzn(z"') 

xpiv„A"y"Z"|L„(fc, a:”, y”, 2”|0 

s 171 E E P2-|L.(^"I0 

' iG£„ z^GCalO. 

Pz"|i,„ (^"I0<® 

X|£„|qzn(z") 

E ^ qzr^iz-) 

iG£„ z"GX>2(0 

= e-”’'^gV (122(0) 

= e-”’'yV ( U 222 ( 0 ) <e 

\iG£„ / 


-ny 


Thus Lemma [T] is proved ■ 

D. Proof of Lemma \2\ 

From Lemma [T] we have the following lemma 
Lemma 5: For any rj > 0 and for any (t/jl"!, , ^ 2 "^) 

satisfying (1/n) log |/C„| > i?i, (1/n) log |£„| > i? 2 , we have 


Ai = 


|/C„||£„ 


E 


E 

„.n 


^ ^E^og; 


VFi(yt|A0 


:-l T't-l'l 


{k,l)&K.„xCn (x",y",z"): 

y"GX>i(fc).z'*Gl> 2(0 
tv" (y ” I x") tV2 (^” |y ”) 

XlJynz"|I,„ (y",2"|0 

X(^(”)(x”|fc,0lFf(y”|x")lF2”(z”|y") 


2?2 < - Elog^ +v\+ 2 e“”’'. 


qzAZt) 


ij-nr/ 


< 


I 2 :,, 


E 


E 1 


(kpeKnXCr, (a:",y".z"): 

y"GX>i(/c),z"GX>2(0 


X(^(”)(x"|fc,Ogy^Z"|L„(y”,^”IO 

E yy"Z"|L„ ( 22 i(fc) X 2 ? 2 ( 0 I 0 


ij-nr/ 


|A 


{k,l)eKnXC„ 


Proof: In ((Sjl in Lemma [T] we choose qz^Y^\Ln 

qY^z^\L„{y^, ^"10 

n 

= {<?yt|£nV‘-iz*-i(ytIOy* ^) 

t=i 

X yzt|L„y‘z*-i(^t|0yO0} 

n 

= Y[UYt\L„Yt-i zt-^{yt\l, y ‘"0 2:*"0l^2(2:t|yt)}- 






















In (|6l) in Lemma [T] we choose having the form 

n 

= X{qzAZt). 

t^l 

Then from the bound (|6]l in Lemma [T] we obtain 




1 "" 

Ri <-Y.^og- 


WiiYt\Xt) 


A r t t t 




by 


U,e-q\K.^) t t 

-^LrrX*Y‘Z* VL 5 i/ ) 


= Ct ^PlA^)vx^lS^ '\l)\{{Wi{y,\x,)W2{z,\y{) 

i^l 

where 

t 

Ct= L! PLr^{l)Pxt\L„{x*\l)Y[{Wi{yi\xi)W2{zi\yi) 

l,x* ,y^ ,z'‘^ i=l 

are constants for normalization. For each t = 1, 2, • • •, n, set 

<^^CZ^=CtCi_\, (24) 

where we define Co = 1. Then we have the following lemma. 


Lemma 6: 


Furthermore, by definition we have 


n 




= logC„,Co = 1. (27) 

From (l26l l and dZTl i. (l25l l is obvious. ■ 

The following lemma is useful for the computation of 
for f = 1, 2, • • •, n. 


^ {Yt\Ln: y‘-i, 2*-i) ^ 

i?2 < - E log J + 2e-"’', 

completing the proof. ■ 

From Lemma |5j we immediately obtain Lemma |2] 

E. Upper Bound of Li^^’^\Wi,W 2 ) 

In this appendix we derive an explicit upper bound of 
Ll^^'^\Wi,W 2 ) to prove Proposition |2] For each t = 
l,2,---,n, define the function of {ut,Xt,yt, Zt) S Ut xX 
x^’ xZ by 

At^.e) , , ^ A W^''iyt\xt)pl^\uS^t\ut) 

4ik .-*(**’= — 0^ ( i ^~e! ^ • 

For each f = 1, 2, • • •, n, we define the probability distribution 

(ll,e-q\U) 

PL„x*y‘z* 


Lemma 7: For each t = l,2,---,n, and for any {I, 
x*,y*,z*) € Cn XT’* xy^ xZ*, we have 

^PXt\L„,xt-^ixt\l,x^~^)Wi{yt\xt)W2{zt\yt) 

Furthermore, we have 


<i> 


itJyS) 
t,q'^ ,K,* 


= E p'l 

Lx* .y* ,2* 


(M.e;g‘-LC"L 


,x‘-iy*-iz‘ 




xpjtdL„,x‘~i(a;t|^a; )VFi(yi|a:i)W^2(zi|yt) 




(29) 


Proof of Lemma [7} By the definition of 
x*,y*, z*), f = 1, 2, • • •, n, we have 

Ufi-XX) (I t t t^ 

PLnX*Y*Z*\LX ^ y , j 

t 

= <7^ Vl„ (Opa‘ I L„ (a::* 10 n {12;*) (2* I Pz) 


Then we have the following chain of equalities; 

t 

(a) ^-1 

i=l 


(30) 


= C^ ^PlA^)px^lS^ *\l)X{{W^{y,\xi)W2{z^yi) 


t 

= C't”VL„(OPA‘-i|L„(a; 

cr^c,_ipi':’^trE;^i 

xpjCt |L„ x*-! (a;t 1 ^ a;‘"L (Pi kt) ^2 (2i |yt) 




Proof: From (l24li we have 




n 

J 2 ^og<^w\ 

t^l 

(25) 




xpjs:t|L„x‘ 

\ogCt-i. 

(26) 



t-i _t-i\ 




..i-l 


( 31 ) 





Steps (a) and (b) follow from (l30l) . From OTl i. we have 

_ fj t-1 t-1 t-iN 

XFXt|L„x‘-i(a:i|^:a:*"^)VFi(yt|a:t)W2(zt|yi) 
p(m.s) 




Taking summations of (l32T i and (l33T l with respect to y*, 
z*, we obtain 

- V (I a-t-i 

xpxt|L„.x‘-i(a^i|^,a:*"^)kFi(yt|xt)fF2(^t|2/i) 

completing the proof. ■ 

We set 

fuS' ■“ ’(“I,a:t) =pK!y.-tz.-Utii,!/*■',z‘“') 


Then by (l29T l in Lemma [T] and the definition of 
{xt,yt,zt \ut), we have 


(P.e) 


fM'"’-. 

= E p'i. 


X 


(ut, a:t) VFi (yt I It) IL2 (zt I y*) 


tXt 

ut,xt,yt,zt 


9yr|;7.(2/t|wt)4t(^t) 


Proof of Proposition 12 is as follows. 
Proof of Proposition^ Set 


A 


Vr,{w^,W2) = {q-.\u\ < \Cr,\\y\^-\ 

qY\x = Wi,qz\Y = -(-T- X o F O F}, 

fll''’^)(fLi,lL 2 ) = _max Z\U). 


q&Vn{Wi,W2) 


We choose g* = qutXtYtZt so that 


qutXtYtZt{ut,xt,yt, zt) 


= Pufxt \ut,xt)Wi{yi\xt)W2{zt\yt)- 


It is obvious that qt £ VniWi, W 2 ) for t = 1, 2, • • •, n. By 
(O and the above choice of qt, we have 


(32) 


$ 


(p.s) 

t,q^ ,K^ 


= <lUtiut)qxt\uA^t\ut)Wi{yt\xt)W2{zt\yt) 


ut,xt,yt,zt 


(33) 


= E, 


Wf{yt\xt) PZt\Vt{zt\vt) \ 

9y,|(7,(2/tkt) qzAzt) I 

W!f{Yt\Xt) Pz,\u,{Zt\Vt) 


Qt 


= E, 


Qt 


flYluS^tlUt) qzAZt) j 
' Wf{Yt\Xt) qz,\uAZt\Ut) Pz,\vAZt\Vt) 

9y|(7t(^*l^*) <lZtiZt) qzt\Ut{Zt\Ut) 


(a 

< E, 


Qt 


WfjYtlXt) qz,\uAZt\Ut) 
qY^luS^tlUt) qzAZt) 


1-9 


X E, 


PZt\vSZt\Vt) 

^ qz.wAZtpt) 

= exp \ (1 - e)n[^f^\XtYtZt\Ut) 

0(^-^)(XtFtFt|C/t) 


(b) 

= exp 

(c) 

< exp 


(d) 

= exp 


1 + A'“«‘ 
1 

TTa“ 

1 

TTa' 


(35) 


Step (a) follows from Holder’s inequality. Step (b) follows 
from (fl^ . Step (c) follows from qt £ 'Pn(Wi,W 2 ) and the 
definition of {Wi, W 2 ). Step (d) follows from Lemma|4] 

in Appendix IaI To prove this lemma we bound the cardinality 
\U\ appearing in the definition of (Wi, W 2 ) to show that 
the bound \IA\ < \X\ is sufficient to describe W 2 ). 

Hence we have the following: 


(34) 


1 


’es" n P 
)(P.9) 




(b) 1 

< 


t=l 


1 +A 


H(^’^)(VFi,VL2). 


(36) 


Step (a) follows from (l25l l in Lemma |2 Step (b) follows from 
(iTSl l. Since (l36l l holds for any n > 1 and any £ 7^^") 
(ILi, kL 2 ), we have 

1 + 7 

Thus, Proposition |2] is proved. ■ 


















